Methods and systems for implementing dynamic properties on objects that support only static properties

ABSTRACT

Methods and systems of simulating dynamic properties on computer-implemented objects that do not support dynamic properties are described. In one embodiment, one or more first objects that do not support dynamic properties are provided. One or more second programmable objects are provided and are programmed to effect property value changes on the objects that do not support dynamic properties. The programmable objects can be programmed using data structures that, in one embodiment, comprise an array of one or more sets of data structures. Each data structure set is associated with a property whose value is desired to be changed. The data structure set can define a new property value, a time at which the property value is to be changed, and how to effect the property value change. The programmable object(s) is pre-programmed with the data structures and knows when to call the first objects so that they can change their properties. In one embodiment, the programmable objects are employed in the context of multi-media project editing software that permits a user to build a multi-media project using multiple different digital source streams.

TECHNICAL FIELD

[0001] This invention generally relates to processing media content and,more particularly, to a system and related interfaces facilitating theprocessing of media content.

BACKGROUND

[0002] Recent advances in computing power and related technology havefostered the development of a new generation of powerful softwareapplications. Gaming applications, communications applications, andmultimedia applications have particularly benefited from increasedprocessing power and clocking speeds. Indeed, once the province ofdedicated, specialty workstations, many personal computing systems nowhave the capacity to receive, process and render multimedia objects(e.g., audio and video content). While the ability to display (receive,process and render) multimedia content has been around for a while, theability for a standard computing system to support true multimediaediting applications is relatively new.

[0003] In an effort to satisfy this need, Microsoft Corporationintroduced an innovative development system supporting advanceduser-defined multimedia editing functions. An example of thisarchitecture is presented in U.S. Pat. No. 5,913, 038 issued toGriffiths and commonly owned by the assignee of the present invention,the disclosure of which is expressly incorporated herein by reference.

[0004] In the '038 patent, Griffiths introduced the an applicationprogram interface which, when exposed to higher-level developmentapplications, enables a user to graphically construct a multimediaprocessing project by piecing together a collection of “filters” exposedby the interface. The interface described therein is referred to as afilter graph manager. The filter graph manager controls the datastructure of the filter graph and the way data moves through the filtergraph. The filter graph manager provides a set of component object model(COM) interfaces for communication between a filter graph and itsapplication. Filters of a filter graph architecture are preferablyimplemented as COM objects, each implementing one or more interfaces,each of which contains a predefined set of functions, called methods.Methods are called by an application program or other component objectsin order to communicate with the object exposing the interface. Theapplication program can also call methods or interfaces exposed by thefilter graph manager object.

[0005] Filter graphs work with data representing a variety of media (ornon-media) data types, each type characterized by a data stream that isprocessed by the filter components comprising the filter graph. A filterpositioned closer to the source of the data is referred to as anupstream filter, while those further down the processing chain isreferred to as a downstream filter. For each data stream that the filterhandles it exposes at least one virtual pin (i.e., distinguished from aphysical pin such as one might find on an integrated circuit). A virtualpin can be implemented as a COM object that represents a point ofconnection for a unidirectional data stream on a filter. Input pinsrepresent inputs and accept data into the filter, while output pinsrepresent outputs and provide data to other filters. Each of the filtersinclude at least one memory buffer, wherein communication of the mediastream between filters is often accomplished by a series of “copy”operations from one filter to another.

[0006] As introduced in Griffiths, a filter graph has three differenttypes of filters: source filters, transform filters, and renderingfilters. A source filter is used to load data from some source; atransform filter processes and passes data; and a rendering filterrenders data to a hardware device or other locations (e.g., saved to afile, etc.). An example of a filter graph for a simplistic mediarendering process is presented with reference to FIG. 1.

[0007]FIG. 1 graphically illustrates an example filter graph forrendering media content. As shown, the filter graph 100 is comprised ofa plurality of filters 102-114, which read, process (transform) andrender media content from a selected source file. As shown, the filtergraph includes each of the types of filters described above,interconnected in a linear fashion.

[0008] Products utilizing the filter graph have been well received inthe market as it has opened the door to multimedia editing usingotherwise standard computing systems. It is to be appreciated, however,that the construction and implementation of the filter graphs arecomputationally intensive and expensive in terms of memory usage. Eventhe most simple of filter graphs requires and abundance of memory tofacilitate the copy operations required to move data between filters.Complex filter graphs can become unwieldy, due in part to the linearnature of prior art filter graph architecture. Moreover, it is to beappreciated that the filter graphs themselves consume memory resources,thereby compounding the issue introduced above.

[0009] Thus, what is required is a filter graph architecture whichreduces the computational and memory resources required to support eventhe most complex of multimedia projects. Indeed, what is required is adynamically reconfigurable multimedia editing system and relatedmethods, unencumbered by the limitations described above. Just such asystem and methods are disclosed below.

SUMMARY

[0010] Methods and systems of simulating dynamic properties oncomputer-implemented objects that do not support dynamic properties aredescribed. In one embodiment, one or more first objects that do notsupport dynamic properties are provided. One or more second programmableobjects are provided and are programmed to effect property value changeson the objects that do not support dynamic properties. The programmableobjects can be programmed using data structures that, in one embodiment,comprise an array of one or more sets of data structures. Each datastructure set is associated with a property whose value is desired to bechanged. The data structure set can define a new property value, a timeat which the property value is to be changed, and how to effect theproperty value change. The programmable object(s) is pre-programmed withthe data structures and knows when to call the first objects so thatthey can change their properties. In one embodiment, the programmableobjects are employed in the context of multi-media project editingsoftware that permits a user to build a multi-media project usingmultiple different digital source streams.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The same reference numbers are used throughout the figures toreference like components and features.

[0012]FIG. 1 is a graphical representation of a conventional filtergraph representing a user-defined development project.

[0013]FIG. 2 is a block diagram of a computing system incorporating theteachings of the described embodiment.

[0014]FIG. 3 is a block diagram of an example software architectureincorporating the teachings of the described embodiment.

[0015]FIG. 4 is a graphical illustration of an example software-enabledmatrix switch, according to an exemplary embodiment.

[0016]FIG. 5 is a graphical representation of a data structurecomprising a programming grid to selectively couple one or more of ascalable plurality of input pins to a scalable plurality of output pinsof the matrix switch filter, in accordance with one aspect of thedescribed embodiment.

[0017]FIG. 6 is a graphical illustration denoting shared buffer memorybetween filters, according to one aspect of the described embodiment.

[0018]FIG. 7 is a flow chart of an example method for generating afilter graph, in accordance with one aspect of the described embodiment.

[0019]FIG. 8 is a flow chart of an example method for negotiating bufferrequirements between at least two adjacent filters, according to oneaspect of the described embodiment.

[0020]FIG. 9 graphically illustrates an overview of a process that takesa user-defined editing project and composites a data structure that canbe used to program the matrix switch.

[0021]FIG. 10 graphically illustrates the project of FIG. 9 in greaterdetail.

[0022]FIG. 11 shows an exemplary matrix switch dynamically generated insupport of the project developed in FIGS. 9 and 10, according to onedescribed embodiment.

[0023]FIG. 12 illustrates a graphic representation of an exemplary datastructure that represents the project of FIG. 10, according to onedescribed embodiment.

[0024] FIGS. 13-18 graphically illustrate various states of a matrixswitch programming grid at select points in processing the project ofFIGS. 9 and 10 through the matrix switch, in accordance with onedescribed embodiment.

[0025]FIG. 19 is a flow chart of an example method for processing mediacontent, in accordance with one described embodiment.

[0026]FIG. 20 illustrates an example project with a transition and aneffect, in accordance with one described embodiment.

[0027]FIG. 21 shows an exemplary data structure in the form of ahierarchical tree that represents the project of FIG. 20.

[0028]FIGS. 22 and 23 graphically illustrate an example matrix switchprogramming grid associated with the project of FIG. 20 at select pointsin time, according to one described embodiment.

[0029]FIG. 24 shows an example matrix switch dynamically generated andconfigured as the grid of FIGS. 22 and 23 was being processed, inaccordance with one described embodiment.

[0030]FIG. 25 shows an exemplary project in accordance with onedescribed embodiment.

[0031]FIG. 26 graphically illustrates an example audio editing project,according to one described embodiment.

[0032]FIG. 27 depicts an example matrix switch programming gridassociated with the project of FIG. 26.

[0033]FIG. 28 shows an example matrix switch dynamically generated andconfigured in accordance with the programming grid of FIG. 27 to performthe project of FIG. 26, according to one described embodiment.

[0034]FIG. 29 illustrates an exemplary media processing projectincorporating another media processing project as a composite, accordingto yet another described embodiment.

[0035]FIG. 30 graphically illustrates an example data structure in theform of a hierarchical tree structure that represents the project ofFIG. 29.

[0036] FIGS. 31-36 graphically illustrate various matrix switchprogramming grid states at select points in generating and configuringthe matrix switch to implement the media processing of FIG. 29.

[0037]FIG. 38 illustrates an example matrix switch suitable for use inthe media processing project of FIG. 29, according to one describedembodiment.

[0038]FIG. 38a graphically illustrates an example data structure in theform of a hierarchical tree structure that represents a project that isuseful in understanding composites in accordance with the describedembodiments.

[0039]FIG. 39 is a flow diagram that describes steps in a method inaccordance with one described embodiment.

DETAILED DESCRIPTION Related Applications

[0040] This application is related to the following commonly-filed U.S.Pat. Applications, all of which are commonly assigned to MicrosoftCorp., the disclosures of which are incorporated by reference herein:

[0041] Application Ser. No.______ , entitled “An Interface and RelatedMethods for Reducing Source Accesses in a Development System”, namingDaniel J. Miller and Eric H. Rudolph as inventors, and bearing attorneydocket number MS1-643US;

[0042] Application Ser. No.______ , entitled “A System and RelatedInterfaces Supporting the Processing of Media Content”, naming Daniel J.Miller and Eric H. Rudolph as inventors, and bearing attorney docketnumber MS1-629US;

[0043] Application Ser. No.______ , entitled “A System and RelatedMethods for Reducing Source Filter Invocation in a Development Project”,naming Daniel J. Miller and Eric H. Rudolph as inventors, and bearingattorney docket number MS1-631US;

[0044] Application Ser. No.______ , entitled “A System and RelatedMethods for Reducing Memory Requirements of a Media Processing System”,naming Daniel J. Miller and Eric H. Rudolph as inventors, and bearingattorney docket number MS1-632US;

[0045] Application Ser. No.______ , entitled “A System and RelatedMethods for Reducing the Instances of Source Files in a Filter Graph”,naming Daniel J. Miller and Eric H. Rudolph as inventors, and bearingattorney docket number MS1-633US;

[0046] Application Ser. No.______ , entitled “An Interface and RelatedMethods for Dynamically Generating a Filter Graph in a DevelopmentSystem”, naming Daniel J. Miller and Eric H. Rudolph as inventors, andbearing attorney docket number MS1-634US;

[0047] Application Ser. No., entitled “A System and Related Methods forProcessing Audio Content in a Filter Graph”, naming Daniel J. Miller andEric H. Rudolph as inventors, and bearing attorney docket numberMS1-639US;

[0048] Application Ser. No.______ , entitled “A System and Methods forGenerating an Managing Filter Strings in a Filter Graph”, naming DanielJ. Miller and Eric H. Rudolph as inventors, and bearing attorney docketnumber MS1-642US;

[0049] Application Ser. No.______ , entitled “Methods and Systems forProcessing Media Content”, naming Daniel J. Miller and Eric H. Rudolphas inventors, and bearing attorney docket number MS1-640US;

[0050] Application Ser. No.______ , entitled “Systems for ManagingMultiple Inputs and Methods and Systems for Processing Media Content ”,naming Daniel J. Miller and Eric H. Rudolph as inventors, and bearingattorney docket number MS1-635US;

[0051] Application Ser. No.______ , entitled “Methods and Systems forImplementing Dynamic Properties on Objects that Support Only StaticProperties”, naming Daniel J. Miller and David Maymudes as inventors,and bearing attorney docket number MS1-63 8US;

[0052] Application Ser. No.______ , entitled “Methods and Systems forEfficiently Processing Compressed and Uncompressed Media Content”,naming Daniel J. Miller and Eric H. Rudolph as inventors, and bearingattorney docket number MS1-630US;

[0053] Application Ser. No.______ , entitled “Methods and Systems forEffecting Video Transitions Represented By Bitmaps”, naming Daniel J.Miller and David Maymudes as inventors, and bearing attorney docketnumber MS1-637US;

[0054] Application Ser. No.______ , entitled “Methods and Systems forMixing Digital Audio Signals”, naming Eric H. Rudolph as inventor, andbearing attorney docket number MS1-636US; and

[0055] Application Ser. No.______, entitled “Methods and Systems forProcessing Multi-media Editing Projects”, naming Eric H. Rudolph asinventor, and bearing attorney docket number MS1-641US.

[0056] Various described embodiments concern an application programinterface associated with a development system. According to one exampleimplementation, the interface is exposed to a media processingapplication to enable a user to dynamically generate complex mediaprocessing tasks, e.g., editing projects. In the discussion herein,aspects of the invention are developed within the general context ofcomputer-executable instructions, such as program modules, beingexecuted by one or more conventional computers. Generally, programmodules include routines, programs, objects, components, datastructures, etc. that perform particular tasks or implement particularabstract data types. Moreover, those skilled in the art will appreciatethat the invention may be practiced with other computer systemconfigurations, including hand-held devices, personal digitalassistants, multiprocessor systems, microprocessor-based or programmableconsumer electronics, network PCs, minicomputers, mainframe computers,and the like. In a distributed computer environment, program modules maybe located in both local and remote memory storage devices. It is noted,however, that modification to the architecture and methods describedherein may well be made without deviating from spirit and scope of thepresent invention. Moreover, although developed within the context of amedia processing system paradigm, those skilled in the art willappreciate, from the discussion to follow, that the application programinterface may well be applied to other development systemimplementations. Thus, the media processing system described below isbut one illustrative implementation of a broader inventive concept.

[0057] Example System Architecture

[0058]FIG. 2 illustrates an example of a suitable computing environment200 on which the system and related methods for processing media contentmay be implemented.

[0059] It is to be appreciated that computing environment 200 is onlyone example of a suitable computing environment and is not intended tosuggest any limitation as to the scope of use or functionality of themedia processing system. Neither should the computing environment 200 beinterpreted as having any dependency or requirement relating to any oneor combination of components illustrated in the exemplary computingenvironment 200.

[0060] The media processing system is operational with numerous othergeneral purpose or special purpose computing system environments orconfigurations. Examples of well known computing systems, environments,and/or configurations that may be suitable for use with the mediaprocessing system include, but are not limited to, personal computers,server computers, thin clients, thick clients, hand-held or laptopdevices, multiprocessor systems, microprocessor-based systems, set topboxes, programmable consumer electronics, network PCs, minicomputers,mainframe computers, distributed computing environments that include anyof the above systems or devices, and the like.

[0061] In certain implementations, the system and related methods forprocessing media content may well be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. The mediaprocessing system may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

[0062] In accordance with the illustrated example embodiment of FIG. 2computing system 200 is shown comprising one or more processors orprocessing units 202, a system memory 204, and a bus 206 that couplesvarious system components including the system memory 204 to theprocessor 202.

[0063] Bus 206 is intended to represent one or more of any of severaltypes of bus structures, including a memory bus or memory controller, aperipheral bus, an accelerated graphics port, and a processor or localbus using any of a variety of bus architectures. By way of example, andnot limitation, such architectures include Industry StandardArchitecture (ISA) bus, Micro Channel Architecture (MCA) bus, EnhancedISA (EISA) bus, Video Electronics Standards Association (VESA) localbus, and Peripheral Component Interconnects (PCI) buss also known asMezzanine bus.

[0064] Computer 200 typically includes a variety of computer readablemedia. Such media may be any available media that is locally and/orremotely accessible by computer 200, and it includes both volatile andnon-volatile media, removable and non-removable media.

[0065] In FIG. 2, the system memory 204 includes computer readable mediain the form of volatile, such as random access memory (RAM) 210, and/ornon-volatile memory, such as read only memory (ROM) 208. A basicinput/output system (BIOS) 212, containing the basic routines that helpto transfer information is between elements within computer 200, such asduring start-up, is stored in ROM 208. RAM 210 typically contains dataand/or program modules that are immediately accessible to and/orpresently be operated on by processing unit(s) 202.

[0066] Computer 200 may further include other removable/non-removable,volatile/non-volatile computer storage media. By way of example only,FIG. 2 illustrates a hard disk drive 228 for reading from and writing toa non-removable, non-volatile magnetic media (not shown and typicallycalled a “hard drive”), a magnetic disk drive 230 for reading from andwriting to a removable, non-volatile magnetic disk 232 (e.g., a “floppydisk”), and an optical disk drive 234 for reading from or writing to aremovable, non-volatile optical disk 236 such as a CD-ROM, DVD-ROM orother optical media. The hard disk drive 228, magnetic disk drive 230,and optical disk drive 234 are each connected to bus 206 by one or moreinterfaces 226.

[0067] The drives and their associated computer-readable media providenonvolatile storage of computer readable instructions, data structures,program modules, and other data for computer 200. Although the exemplaryenvironment described herein employs a hard disk 228, a removablemagnetic disk 232 and a removable optical disk 236, it should beappreciated by those skilled in the art that other types of computerreadable media which can store data that is accessible by a computer,such as magnetic cassettes, flash memory cards, digital video disks,random access memories (RAMs), read only memories (ROM), and the like,may also be used in the exemplary operating environment.

[0068] A number of program modules may be stored on the hard disk 228,magnetic disk 232, optical disk 236, ROM 208, or RAM 210, including, byway of example, and not limitation, an operating system 214, one or moreapplication programs 216 (e.g., multimedia application program 224),other program modules 218, and program data 220. In accordance with theillustrated example embodiment of FIG. 2, operating system 214 includesan application program interface embodied as a render engine 222. Aswill be developed more fully below, render engine 222 is exposed tohigher-level applications (e.g., 216) to automatically assemble filtergraphs in support of user-defined development projects, e.g., mediaprocessing projects. Unlike conventional media processing systems,however, render engine 222 utilizes a scalable, dynamicallyreconfigurable matrix switch to reduce filter graph complexity, therebyreducing the computational and memory resources required to complete adevelopment project. Various aspects of the innovative media processingsystem represented by a computer 200 implementing the innovative renderengine 222 will be developed further, below.

[0069] Continuing with FIG. 2, a user may enter commands and informationinto computer 200 through input devices such as keyboard 238 andpointing device 240 (such as a “mouse”). Other input devices may includea audio/video input device(s) 253, a microphone, joystick, game pad,satellite dish, serial port, scanner, or the like (not shown). These andother input devices are connected to the processing unit(s) 202 throughinput interface(s) 242 that is coupled to bus 206, but may be connectedby other interface and bus structures, such as a parallel port, gameport, or a universal serial bus (USB).

[0070] A monitor 256 or other type of display device is also connectedto bus 206 via an interface, such as a video adapter 244. In addition tothe monitor, personal computers typically include other peripheraloutput devices (not shown), such as speakers and printers, which may beconnected through output peripheral interface 246.

[0071] Computer 200 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer250. Remote computer 250 may include many or all of the elements andfeatures described herein relative to computer 200 including, forexample, render engine 222 and one or more development applications 216utilizing the resources of render engine 222.

[0072] As shown in FIG. 2. computing system 200 is communicativelycoupled to remote devices (e.g., remote computer 250) through a localarea network (LAN) 251 and a general wide area network (WAN) 252. Suchnetworking environments are commonplace in offices, enterprise-widecomputer networks, intranets, and the Internet.

[0073] When used in a LAN networking environment, the computer 200 isconnected to LAN 251 through a suitable network interface or adapter248. When used in a WAN networking environment, the computer 200typically includes a modem 254 or other means for establishingcommunications over the WAN 252. The modem 254, which may be internal orexternal, may be connected to the system bus 206 via the user inputinterface 242, or other appropriate mechanism.

[0074] In a networked environment, program modules depicted relative tothe personal computer 200, or portions thereof, may be stored in aremote memory storage device. By way of example, and not limitation,FIG. 2 illustrates remote application programs 216 as residing on amemory device of remote computer 250. It will be appreciated that thenetwork connections shown and described are exemplary and other means ofestablishing a communications link between the computers may be used.

[0075] Turning next to FIG. 3, a block diagram of an example developmentsystem architecture is presented, in accordance with one embodiment ofthe present invention. In accordance with the illustrated exampleembodiment of FIG. 3, development system 300 is shown comprising one ormore application program(s) 216 coupled to render engine 222 via anappropriate communications interface 302. As used herein, applicationprogram(s) 216 are intended to represent any of a wide variety ofapplications which may benefit from use of render engine 222 such as,for example a media processing application 224.

[0076] The communications interface 302 is intended to represent any ofa number of alternate interfaces used by operating systems to exposeapplication program interface(s) to applications. According to oneexample implementation, interface 302 is a component object model (COM)interface, as used by operating systems offered by MicrosoftCorporation. As introduced above, COM interface 302 provides a means bywhich the features of the render engine 222, to be described more fullybelow, are exposed to an application program 216.

[0077] In accordance with the illustrated example implementation of FIG.3, render engine 222 is presented comprising source filter(s) 304A-N,transform filter(s) 306A-N and render filter 310, coupled togetherutilizing virtual pins to facilitate a user-defined media processingproject. According to one implementation, the filters of system 300 aresimilar to the filters exposed in conventional media processing systems.According to one implementation, however, filters are not coupled viasuch interface pins. Rather, alternate implementations are envisionedwherein individual filters (implemented as objects) make calls to otherobjects, under the control of the render engine 222, for the desiredinput. Unlike conventional systems, however, render engine 222 exposes ascalable, dynamically reconfigurable matrix switch filter 308,automatically generated and dynamically configured by render engine 222to reduce the computational and memory resource requirements oftenassociated with development projects. As introduced above, the pins(input and/or output) are application interface(s) designed tocommunicatively couple other objects (e.g., filters).

[0078] In accordance with the example implementation of a mediaprocessing system, an application communicates with an instance ofrender engine 222 when the application 216 wants to process streamingmedia content. Render engine 222 selectively invokes and controls aninstance of filter graph manager (not shown) to automatically create afilter graph by invoking the appropriate filters (e.g., source,transform and rendering).As introduced above, the communication of mediacontent between filters is achieved by either (1) coupling virtualoutput pins of one filter to the virtual input pins of requestingfilter; or (2) by scheduling object calls between appropriate filters tocommunicate the requested information. As shown, source filter 304receives streaming data from the invoking application or an externalsource (not shown). It is to be appreciated that the streaming data canbe obtained from a file on a disk, a network, a satellite feed, anInternet server, a video cassette recorder, or other source of mediacontent. As introduced above, transform filter(s) 306 take the mediacontent and processes it in some manner, before passing it along torender filter 310. As used herein, transform filter(s) 306 are intendedto represent a wide variety of processing methods or applications thatcan be performed on media content. In this regard, transform filter(s)306 may well include a splitter, a decoder, a sizing filter, atransition filter, an effects filter, and the like. The function of eachof these filters is described more fully in the Griffiths application,introduced above, and generally incorporated herein by reference. Thetransition filter, as used herein, is utilized by render engine 222 totransition the rendered output from a first source to a second source.The effect filter is selectively invoked to introduce a particulareffect (e.g., fade, wipe, audio distortion, etc.) to a media stream.

[0079] In accordance with one aspect of the embodiment, to be describedmore fully below, matrix switch filter 308 selectively passes mediacontent from one or more of a scalable plurality of input(s) to ascalable plurality of output(s). Moreover, matrix switch 308 alsosupports implementation of a cascaded architecture utilizing feedbackpaths, i.e., wherein transform filters 306B, 306C, etc. coupled to theoutput of matrix switch 308 are dynamically coupled to one or more ofthe scalable plurality of matrix switch input(s). An example of thiscascaded filter graph architecture is introduced in FIG. 3, and furtherexplained in example implementations, below.

[0080] Typically, media processed through source, transform and matrixswitch filters are ultimately passed to render filter 310, whichprovides the necessary interface to a hardware device, or other locationthat accepts the renderer output format, such as a memory or disk file,or a rendering device.

[0081]FIG. 4 is a graphical illustration of an example software-enabledmatrix switch 308, according to one example embodiment of the presentinvention. As shown, the matrix switch 308 is comprised of a scalableplurality of input(s) 402 and a scalable plurality of output(s) 404,wherein any one or more of the input(s) 402 may be iteratively coupledto any one or more of the output(s) 404, based on the content of thematrix switch programming grid 406, automatically generated by renderengine 222. According to an alternate implementation introduced above,switch matrix 308 is programmed by render engine 222 to dynamicallygenerate object calls to communicate media content between filters. Inaddition, according to one implementation, matrix switch 308 includes aplurality of input/output (I/O) buffers 408, as well as means formaintaining source, or media time 410 and/or timeline, or project time412. It is to be appreciated, however, that in alternate implementationsmatrix switch 308 does not maintain both source and project times,relying on an upstream filter to convert between these times. As will bedeveloped more fully below, matrix switch 308 dynamically couples one ormore of the scalable plurality of inputs 402 to one or more of thescalable plurality of outputs 404 based, at least in part, on the mediatime 410 and/or the project time 412 and the content of matrix switchprogramming grid 406. In this regard, matrix switch 308 may becharacterized as time-aware, supporting such advanced editing featuresas searching/seeking to a particular point (e.g., media time) in themedia content, facilitating an innovative buffering process utilizingI/O buffers 408 to facilitate look-ahead processing of media content,and the like. Thus, it will be appreciated given the discussion tofollow that introduction of the matrix switch 308 provides a user withan editing flexibility that was heretofore unavailable in a personalcomputer-based media processing system.

[0082] As introduced above, the inputs 402 and outputs 404 of matrixswitch 308 are interfaces which facilitate the time-sensitive routing ofdata (e.g., media content) in accordance with a user-defined developmentproject. Matrix switch 308 has a scalable plurality of inputs 402 andoutputs 404, meaning that the number of inputs 402 and outputs 404 areindividually generated to satisfy a given editing project. Insofar aseach of the inputs/outputs (I/O) has an associated transfer buffer(preferably shared with an adjacent filter) to communicate mediacontent, the scalability of the input/output serves to reduce theoverall buffer memory consumed by an editing project. According to oneimplementation, output 1 is generally reserved as a primary output,e.g., coupled to a rendering filter (not shown).

[0083] According to one implementation, for each input 402 and output404, matrix switch 308 attempts to be the allocator, or manager of thebuffer associated with the I/O(s) shared with adjacent filters. Onereason is to ensure that all of the buffers are of the same size andshare common attributes so that a buffer associated with any input 402may be shared with any output 404, thereby reducing the need to copymemory contents between individual buffers associated with suchinputs/outputs. If matrix switch 308 cannot be an allocator for a givenoutput (404), communication from an input (402) to that output isperformed using a conventional memory copy operation between theindividual buffers associated with the select input/output.

[0084] As introduced above, the matrix switch programming grid 406 isdynamically generated by render engine 222 based, at least in part, onthe user-defined development project. As will be developed below, renderengine 222 invokes an instance of filter graph manager to assembles atree structure of an editing project, noting dependencies betweensource, filters and time to dynamically generate the programming grid406. A data structure comprising an example programming grid 406 isintroduced with reference to FIG. 5, below.

[0085] Turning briefly to FIG. 5, a graphical representation of a datastructure comprising an example programming grid 406 is presented, inaccordance with one embodiment of the present invention. In accordancewith the illustrated example embodiment of FIG. 5, programming grid 406is depicted as a two-dimensional data structure comprising a columnalong the y-axis 502 of the grid denoting input pins associated with acontent chain (e.g., series of filters to process media content) of thedevelopment project. The top row along the x-axis 504 of the datastructure denotes project time. With these grid “borders”, the body 506of the grid 406 is populated with output pin assignments, denoting whichinput pin is coupled to which output pin during execution of thedevelopment project. In this way, render engine 222 dynamicallygenerates and facilitates matrix switch 308. Those skilled in the artwill appreciate, however, that data structures of greater or lessercomplexity may well be used in support of the programming grid 406without deviating from the spirit and scope of the present invention.

[0086] Returning to FIG. 4, matrix switch 308 is also depicted with aplurality of input/output buffers 408, shared among all of theinput(s)/output(s) (402, 404) to facilitate advanced processingfeatures. That is, while not required to implement the core features ofmatrix switch 308, I/O buffers 408 facilitate a number of innovativeperformance enhancing features to improve the performance (or at leastthe user's perception of performance) of the processing system, therebyproviding an improved user experience. According to one implementation,I/O buffers 408 are separate from the buffers assigned to eachindividual input and output pin in support of communication through theswitch. According to one implementation, I/O buffers 408 are primarilyused to foster look-ahead processing of the project. Assume, forexample, that a large portion of the media processing project requiredonly 50% of the available processing power, while some smaller portionrequired 150% of the available processing power. Implementation of theshared I/O buffers 408 enable filter graph manager to execute tasksahead of schedule and buffer this content in the shared I/O buffers 408until required. Thus, when execution of the filter graph reaches a pointwhere more than 100% of the available processing power is required, theprocessing system can continue to supply content from the I/O buffers408, while the system completes execution of the CPU-intensive tasks. Ifenough shared buffer space is provided, the user should never know thatsome tasks were not performed in real-time. According to oneimplementation, shared buffers 408 are dynamically split into two groupsby render engine 222, a first group supports the input(s) 402, while asecond (often smaller) group is used in support of a primary output(e.g., output pin 1) to facilitate a second, independent outputprocessing thread. The use of an independent output buffers the renderengine from processing delays that might occur in upstream and/ordownstream filters, as discussed above. It will be appreciated by thoseskilled in the art that such that matrix switch 308 and the foregoingdescribed architecture beneficially suited to support media streamingapplications.

[0087] As introduced above, the filter graph is time-aware in the sensethat media (source) time and project execution time are maintained.According to one implementation, matrix switch 308 maintains at leastthe project clock, while an upstream filter maintains the source time,converting between source and project time for all downstream filters(i.e., including the matrix switch 308). According to oneimplementation, the frame rate converter filter of a filter graph isresponsible for converting source time to project time, and vice versa,i.e., supporting random seeks, etc. Alternatively, matrix switch 308utilizes an integrated set of clock(s) to independently maintain projectand media times.

[0088] Having introduced the architectural and operational elements ofmatrix switch filter 308, FIG. 6 graphically illustrates an examplefilter graph implementation incorporating the innovative matrix switch308. In accordance with the illustrated example embodiment, filter graph600 is generated by render engine 222 in response to a user defineddevelopment project. Unlike the lengthy linear filter graphs typical ofconvention development systems however, filter graph 600 is shownincorporating a matrix switch filter 308 to recursively route thepre-processed content (e.g., through filters 602, 606, 610, 614 and 618,described more fully below) through a user-defined number of transformfilters including, for example, transition filter(s) 620 and effectsfilter(s) 622. Moreover, as will be developed more fully below, thescalable nature of matrix switch filter 308 facilitates such iterativeprocessing for any number of content threads, tracks or compositions.

[0089] According to one implementation, a matrix switch filter 308 canonly process one type of media content, of the same size and at the sameframe-rate (video) or modulation type/schema (audio). Thus, FIG. 6 isdepicted comprising pre-processing filters with a parser filter 606 toseparate, independent content type(s) (e.g., audio content and videocontent), wherein one of the media types would be processed along adifferent path including a separate instance of matrix switch 308. Thus,in accordance with the illustrated example embodiment of a mediaprocessing system, processing multimedia content including audio andvideo would utilize two (2) matrix switch filters 308, one dedicated toaudio processing (not shown) and one dedicated to video processing. Thatis not to say, however, that multiple switch filters 308 could not beused (e.g., two each for audio and video) for each content type inalternate implementations. Similarly, it is anticipated that inalternate implementations a matrix switch 308 that accepts multiplemedia types could well be used without deviating from the spirit andscope of the present invention.

[0090] In addition filter graph 600 includes a decoder filter 610 todecode the media content. Resize filter 614 is employed when matrixswitch 308 is to receive content from multiple sources, ensuring thatthe size of the received content is the same, regardless of the source.According to one implementation, resize filter 614 is selectivelyemployed in video processing paths to adjust the media size of contentfrom one or more sources to a user-defined level. Alternatively, resizerfilter 614 adjusts the media size to the largest size provided by anyone or more media sources. That is, if, for example, render engine 222identifies the largest required media size (e.g., 1270×1040 video pixelsper frame) and, for any content source not providing content at thissize, the content is modified (e.g., stretched, packed, etc.) to fillthis size requirement. The frame rate converter (FRC) and pack filter618, introduced above, ensures that video content from the multiplesources is arriving at the same frame rate, e.g., ten (10) frames persecond. As introduced above, the FRC also maintains the distinctionbetween source time and project time.

[0091] In accordance with one aspect of the present invention, filtergraph 600 is depicted utilizing a single, negotiated buffer 604, 608,612, 616, etc. between adjacent filters. In this regard, render engine222 reduces the buffer memory requirements in support of a developmentproject.

[0092] From the point of pre-processing (filters 602, 606, 610, 614,618), rather than continue a linear filter graph incorporating all ofthe transition 620 and effect 622 filter(s), render engine 222 utilizesa cascade architecture, recursively passing media content through thematrix switch 308 to apply to the transform filter(s) (e.g., 620, 622,etc.) to complete the execution of the development project. It will beappreciated by those skilled in the art that the ability to recursivelypass media content through one or more effect and/or transition filtersprovided by the matrix switch filter 308 greatly reduces the perceivedcomplexity of otherwise large filter graphs, while reducing memory andcomputational overhead.

[0093] Turning to FIG. 7, a flow chart of an example method forgenerating a filter graph is presented, in accordance with one aspect ofthe present invention. The method 700 begins with block 702 whereinrender engine 222 receives an indication to generate a filter graphrepresenting a user-defined development project (e.g., a media editingproject). According to one example implementation, the indication isreceived from an application 224 via COM interface(s) 302.

[0094] In block 704, render engine 222 facilitates generation of theediting project, identifying the number and type of media sourcesselected by the user. In block 706, based at least in part on the numberand/or type of media sources, filter graph manger 222 exposes source,transform and rendering filter(s) to effect a user defined mediaprocessing project, while beginning to establish a programming grid 406for the matrix switch filter 308.

[0095] In block 708, reflecting user editing instructions, render engine222 completes the programming grid 406 for matrix switch 308,identifying which inputs 402 are to be coupled to which outputs 404 atparticular project times.

[0096] Based, at least in part, on the programming grid 406 renderengine 222 generates a matrix switch filter 308 with an appropriatenumber of input 402 and output 404 pins to effect the project, andassembles the filter graph, block 710.

[0097] In block 712, to reduce the buffer memory requirements for theprocessing project, the render engine 222 instructs the filterspopulating the filter graph to (re)negotiate buffer memory requirementsbetween filters. That is, adjacent filters attempt to negotiate a sizeand attribute standard so that a single buffer can be utilized to coupleeach an output pin of one filter to an input pin of a downstream filter.An example implementation of the buffer negotiation process of block 712is presented in greater detail with reference to FIG. 8.

[0098] Turning briefly to FIG. 8, an example method of negotiatingbuffer requirements between adjacent filters is presented, in accordancewith one example implementation of the present invention. Once the finalconnection is established to matrix switch 308, matrix switch 308identifies the maximum buffer requirements for any filter coupled to anyof its pins (input 402 and/or output 404), block 802. According to oneimplementation, the maximum buffer requirements are defined as thelowest common multiple of buffer alignment requirements, and the maximumof all the pre-fix requirements of the filter buffers.

[0099] In block 804, matrix switch 308 selectively removes one or moreexisting filter connections to adjacent filters. Matrix switch 308 thenreconnects all of its pins to adjacent filters using a common buffersize between each of the pins, block 806. In block 808, matrix switch308 negotiates to be the allocator for all of its pins (402, 404). Ifthe matrix switch 308 cannot, for whatever reason, be the allocator forany of its input pins 402 minimal loss to performance is encountered, asthe buffer associated with the input pin will still be compatible withany downstream filter (i.e., coupled to an output pin) and, thus, thebuffer can still be passed to the downstream filter without requiring amemory copy operation. If, however, matrix switch 308 cannot be anallocator for one of its output pins 404, media content must then betransferred to at least the downstream filter associated with thatoutput pin using a memory copy operation, block 810.

[0100] In block 812, once the matrix switch 308 has re-established itsconnection to adjacent filters, render engine 222 restores theconnection in remaining filters using negotiated buffer requirementsemanating from the matrix switch filter 308 buffer negotiations. Oncethe connections throughout the filter graph have been reconnected, theprocess continues with block 714 of FIG. 7.

[0101] In block 714 (FIG. 7), have re-established the connectionsbetween filters, render engine 222 is ready to implement a user'sinstruction to execute the media processing project.

[0102] Example Operation and Implementation(s)

[0103] The matrix switch described above is quite useful in that itallows multiple inputs to be directed to multiple outputs at any onetime. These input can compete for a matrix switch output. Theembodiments described below permit these competing inputs to beorganized so that the inputs smoothly flow through the matrix switch toprovide a desired output. And, while the inventive programmingtechniques are described in connection with the matrix switch as such isemployed in the context of multi-media editing projects, it should beclearly understood that application of the inventive programmingtechniques and structures should not be so limited only to applicationin the field of multi-media editing projects or, for that matter,multi-media applications or data streams. Accordingly, the principlesabout to be discussed can be applied to other fields of endeavor inwhich multiple inputs can be characterized as competing for a particularoutput during a common time period.

[0104] In the multi-media example below, the primary output of thematrix switch is a data stream that defines an editing project that hasbeen created by a user. Recall that this editing project can includemultiple different sources that are combined in any number of differentways, and the sources that make up a project can comprise audio sources,video sources, or both. The organization of the inputs and outputs ofthe matrix switch are made manageable, in the examples described below,by a data structure that permits the matrix switch to be programmed.

[0105]FIG. 9 shows an overview of a process that takes a user-definedediting project and renders from it a data structure that can be used toprogram the matrix switch.

[0106] Specifically, a user-defined editing project is shown generallyat 900. Typically, when a user creates an editing project, they canselect from a number of different multimedia clips that they can thenassemble into a unique presentation. Each individual clip represents asource of digital data or a source stream (e.g., multimedia content).Projects can include one or more sources 902. In defining their project,a user can operate on sources in different ways. For example, videosources can have transitions 904 and effects 906 applied on them. Atransition object is a way to change between two or more sources. Asdiscussed above, a transition essentially receives as input, two or morestreams, operates on them in some way, and produces a single outputstream. An exemplary transition can comprise, for example, fading fromone source to another. An effect object can operate on a single sourceor on a composite of sources. An effect essentially receives a singleinput stream, operates on it in some way, and produces a single outputstream. An exemplary effect can comprise a black-and-white effect inwhich a video stream that is configured for presentation in color formatis rendered into a video stream that is configured for presentation inblack and white format. Unlike conventional effect filters, effectobject 906 may well perform multiple effect tasks. That is, inaccordance with one implementation, an effect object (e.g., 906) mayactually perform multiple tasks on the received input stream, whereinsaid tasks would require multiple effect filters in a conventionalfilter graph system.

[0107] An exemplary user interface 908 is shown and represents what auser might see when they produce a multimedia project with softwareexecuting on a computer. In this example, the user has selected threesources A, B, and C, and has assembled the sources into a projecttimeline. The project timeline defines when the individual sources areto be rendered, as well as when any transitions and/or effects are tooccur.

[0108] In the discussion that follows, the notion of a track isintroduced. A track can contain one or more sources or source clips. Ifa track contains more than one source clip, the source clips cannotoverlap. If source clips are to overlap (e.g. fading from one source toanother, or having one source obscure another), then multiple tracks areused. A track can thus logically represent a layer on which sequentialvideo is produced. User interface 908 illustrates a project thatutilizes three tracks, each of which contains a different source. Inthis particular project source A will show for a period of time. At adefined time in the presentation, source A is obscured by source B. Atsome later time, source B transitions to source C.

[0109] In accordance with the described embodiment, the user-definedediting project 900 is translated into a data structure 910 thatrepresents the project. In the illustrated and described example, thisdata structure 910 comprises a tree structure. It is to be understood,however, that other data structures could be used. The use of treestructures to represent editing projects is well-known and is notdescribed here in any additional detail. Once the data structure 910 isdefined, it is processed to provide a data structure 912 that isutilized to program the matrix switch. In the illustrated and describedembodiment, data structure 912 comprises a grid from which the matrixswitch can be programmed. It is to be understood and appreciated thatother data structures and techniques could, however, be used to programthe matrix switch without departing from the spirit and scope of theclaimed subject matter.

[0110] The processing that takes place to define data structures 910 and912 can take place using any suitable hardware, software, firmware, orcombination thereof. In the examples set forth below, the processingtakes place utilizing software in the form of a video editing softwarepackage that is executable on a general purpose computer.

[0111] Example Project

[0112] For purposes of explanation, consider FIG. 10 which shows project908 from FIG. 9 in a little additional detail. Here, a time linecontaining numbers 0-16 is provided adjacent the project to indicatewhen particular sources are to be seen and when transitions and effects(when present) are to occur. In the examples in this document, thefollowing convention exists with respect to projects, such as project908. A priority exists for video portions of the project such that asone proceeds from top to bottom, the priority increases. Thus, in theFIG. 10 example, source A has the lowest priority followed by source Band source C. Thus, if there is an overlap between higher and lowerpriority sources, the higher priority source will prevail. For example,source B will obscure source A from between t=4-8.

[0113] In this example, the following can be ascertained from theproject 908 and time line: from time t=0-4 source A should be routed tothe matrix switch's primary output; from t=4-12 source B should berouted to the matrix switch's primary output; from t=12-14 there shouldbe a transition between source B and source C which should be routed tothe matrix switch's primary output; and from t=14-16 source C should berouted to the matrix switch's primary output. Thus, relative to thematrix switch, each of the sources and the transition can becharacterized by where it is to be routed at any given time. Consider,for example, the table just below: Object Routing for a given time C t =0-12 (nowhere); t = 12-14 (transition); t = 14-16 (primary output) B t =0-4 (nowhere); t = 4-12 (primary output); t = 12-14 (transition); t =14-16 (nowhere) A t = 0-4 (primary output); t = 4-16 (nowhere)Transition t = 0-12 (nowhere); t = 12-14 (primary output); t = 14-16(nowhere)

[0114]FIG. 11 shows an exemplary matrix switch 1100 that can be utilizedin the presentation of the user's project. Matrix switch 1100 comprisesmultiple inputs and multiple outputs. Recall that a characteristic ofthe matrix switch 1100 is that any of the inputs can be routed to any ofthe outputs at any given time. A transition element 1102 is provided andrepresents the transition that is to occur between sources B and C.Notice that the matrix switch includes four inputs numbered 0-3 andthree outputs numbered 0-2. Inputs 0-2 correspond respectively tosources A-C, while input 3 corresponds to the output of the transitionelement 1102. Output 0 corresponds to the switch's primary output, whileoutputs 1 and 2 are routed to the transition element 1102.

[0115] The information that is contained in the table above is theinformation that is utilized to program the matrix switch. Thediscussion presented below describes but one implementation in which theinformation contained in the above table can be derived from the user'sproject time line.

[0116] Recall that as a user edits or creates a project, software thatcomprises a part of their editing software builds a data structure thatrepresents the project. In the FIG. 9 overview, this was data structure910. In addition to building the data structure that represents theediting project, the software also builds and configures a matrix switchthat is to be used to define the output stream that embodies theproject. Building and configuring the matrix switch can include buildingthe appropriate graphs (e.g., a collection of software objects, orfilters) that are associated with each of the sources and associatingthose graphs with the correct inputs of the matrix switch. In addition,building and configuring the matrix switch can also include obtainingand incorporating additional appropriate filters with the matrix switch,e.g. filters for transitions, effects, and mixing (for audio streams).This will become more apparent below.

[0117]FIG. 12 shows a graphic representation of an exemplary datastructure 1200 that represents the project of FIG. 10. Here, the datastructure comprises a traditional hierarchical tree structure. Anysuitable data structure can, however, be utilized. The top node 1202constitutes a group node. A group encapsulates a type of media. Forexample, in the present example the media type comprises video. Anothermedia type is audio. The group node can have child nodes that are eithertracks or composites. In the present example, three track nodes 1204,1206, and 1208 are shown. Recall that each track can have one or moresources. If a track comprises more than one source, the sources cannotoverlap. Here, all of the sources (A, B, and C) overlap. Hence, threedifferent tracks are utilized for the sources. In terms of priority, thelowest priority source is placed into the tree furthest from the left at1204 a. The other sources are similarly placed. Notice that source C(1208 a) has a transition 1210 associated with it. A transition object,in this example, defines a two-input/one output operation. When appliedto a track or a composition (discussed below in more detail), thetransition object will operate between the track to which it has beenapplied, and any objects that are beneath it in priority and at the samelevel in the tree. A “tree level” has a common depth within the tree andbelongs to the same parent. Accordingly, in this example, the transition1210 will operate on a source to the left of the track on which source Cresides, and beneath it in priority, i.e. source B. If the transition isapplied to any object that has nothing beneath it in the tree, it willtransition from blackness (and/or silence if audio is included).

[0118] Once a data structure representing the project has been built, inthis case a hierarchical tree structure, a rendering engine processesthe data structure to provide another data structure that is utilized toprogram the matrix switch. In the FIG. 9 example, this additional datastructure is represented at 912. It will be appreciated and understoodthat the nodes of tree 1200 can include so-called meta information suchas a name, ID, and a time value that represents when that particularnode's object desires to be routed to the output, e.g. node 1204 a wouldinclude an identifier for the node associating it with source A, as wellas a time value that indicates that source A desires to be routed to theoutput from time t=0-8. This meta information is utilized to build thedata structure that is, in turn, utilized to program the matrix switch.

[0119] In the example about to be described below, a specific datastructure in the form of a grid is utilized. In addition, certainspecifics are described with respect to how the grid is processed sothat the matrix switch can be programmed. It is to be understood thatthe specific described approach is for exemplary purposes only and isnot intended to limit application of the claims. Rather, the specificapproach constitutes but one way of implementing broader conceptualnotions embodied by the inventive subject matter.

[0120] FIGS. 13-18 represent a process through which the inventive gridis built. In the grid about to be described, the x axis represents time,and the y axis represents layers in terms of priority that go fromlowest (at the top of the grid) to highest (at the bottom of the grid).Every row in the grid represents the video layer. Additionally, entriesmade within the grid represent output pins of the matrix switch. Thiswill become apparent below.

[0121] The way that the grid is built in this example is that therendering engine does a traversal operation on the tree 1200. In thisparticular example, the traversal operation is known as a “depth-first,left-to-right” traversal. This operation will layerize the nodes so thatthe leftmost track or source has the lowest priority and so on. Doingthe above-mentioned traversal on tree 1200 (FIG. 12), the first nodeencountered is node 1204 which is associated with source A. This is thelowest priority track or source. A first row is defined for the grid andis associated with source A. After the first grid row is defined, a gridentry is made and represents the time period for which source A desiresto be routed to the matrix switch's primary output.

[0122]FIG. 13 shows the state of a grid 1300 after this first processingstep. Notice that from time t=0-8, a “0” has been placed in the grid.The “0” represents the output pin of the matrix switch—in this case theprimary output. Next, the traversal encounters node 1206 (FIG. 12) whichis associated with source B. A second row is thus defined for the gridand is associated with source B. After the second grid row is defined, agrid entry is made and represents the time period for which source Bdesires to be routed to the matrix switch's primary output.

[0123]FIG. 14 shows the state of grid 1300 after this second processingstep. Notice that from time t=4-14, a “0” has been placed in the grid.Notice at this point that something interesting has occurred which willbe resolved below. Each of the layers has a common period of time (i.e.t=4-8) for which it desires to be routed to the matrix switch's primaryoutput. However, because of the nature of the matrix switch, only oneinput can be routed to the primary output at a time. Next, the traversalencounters node 1208 (FIG. 12) which is associated with source C. Inthis particular processing example, a rule is defined that sources ontracks are processed before transitions on the tracks are processedbecause transitions operate on two objects that are beneath them. Athird row is thus defined for the grid and is associated with source C.After the third row is defined, a grid entry is made and represents thetime period for which source C desires to be routed to the matrixswitch's primary output.

[0124]FIG. 15 shows the state of grid 1300 after this third processingstep. Notice that from time t=12-16, a “0” has been placed in the grid.Next, the traversal encounters node 1210 (FIG. 12) which corresponds tothe transition. Thus, a fourth row is defined in the grid and isassociated with the transition. After the fourth row is defined, a gridentry is made and represents the time period for which the transitiondesires to be routed to the matrix switch's primary output.

[0125]FIG. 16 shows the state of grid 1300 after this fourth processingstep. Notice that from time t=12-14, a “0” has been placed in the gridfor the transition entry. The transition is a special grid entry. Recallthat the transition is programmed to operate on two inputs and provide asingle output. Accordingly, starting at the transition entry in the gridand working backward, each of the entries corresponding to the same treelevel are examined to ascertain whether they contain entries thatindicate that they want to be routed to the output during the same timethat the transition is to be routed to the output. If grid entries arefound that conflict with the transition's grid entry, the conflictinggrid entry is changed to a value to corresponds to an output pin thatserves as an input to the transition element 1102 (FIG. 11). This isessentially a redirection operation. In the illustrated grid example,the transition first finds the level that corresponds to source C. Thislevel conflicts with the transition's grid entry for the time periodt=12-14. Thus, for this time period, the grid entry for level C ischanged to a switch output that corresponds to an input for thetransition element. In this example, a “2” is placed in the grid tosignify that for this given time period, this input is routed to outputpin 2. Similarly, continuing up the grid, the next level that conflictswith the transition's grid entry is the level that corresponds to sourceB. Thus, for the conflicting time period, the grid entry for level B ischanged to a switch output that corresponds to an input for thetransition element. In this example, a “1” is placed in the grid tosignify that for this given time period, this input is routed to outputpin 1 of the matrix switch.

[0126]FIG. 17 shows the state of the grid at this point in theprocessing. Next, a pruning function is implemented which removes anyother lower priority entry that is contending for the output with ahigher priority entry. In the example, the portion of A from t=4-8 getsremoved because the higher priority B wants the output for that time.

[0127]FIG. 18 shows the grid with a cross-hatched area that signifiesthat portion of A's grid entry that has been removed.

[0128] At this point, the grid is in a state in which it can be used toprogram the matrix switch. The left side entries—A, B, C, and TRANSrepresent input pin numbers 0, 1, 2, and 3 (as shown) respectively, onthe matrix switch shown in FIG. 11. The output pin numbers of the matrixswitch are designated at 0, 1, and 2 both on the switch in FIG. 11 andwithin the grid in FIG. 18. As one proceeds through the grid, startingwith source A, the programming of the matrix switch can be ascertainedas follows: A is routed to output pin 0 of the matrix switch (theprimary output) from t=0-4. From t=4-16, A is not routed to any outputpins. From t=0-4, B is not routed to any of the output pins of thematrix switch. From t =4-12, B is routed to the primary output pin 0 ofthe matrix switch. From t=12-14, B is routed to output pin 1 of thematrix switch. Output pin 1 of the matrix switch corresponds to one ofthe input pins for the transition element 1102 (FIG. 11). From t=14-16,B is not routed to any of the output pins of the matrix switch. Fromt−0-12, C is not routed to any of the output pins of the matrix switch.From t=12-14, C is routed to output pin 2 of the matrix switch. Outputpin 2 of the matrix switch corresponds to one of the input pins for thetransition element 302 (FIG. 3). From t=12-14 the transition element(input pin 3) is routed to output pin 0. From t=14-16, C is routed tooutput pin 0 of the matrix switch.

[0129] As alluded to above, one of the innovative aspects of the matrixswitch 308 is its ability to seek to any point in a source, withouthaving to process the intervening content serially through the filter.Rather, matrix switch 308 identifies an appropriate transition point anddumps at least a subset of the intervening content, and continuesprocessing from the seeked point in the content.

[0130] The ability of the matrix switch 308 to seek to any point in themedia content gives rise to certain performance enhancement heretoforeunavailable in computer implemented media processing systems. Forexample, generation of a filter graph by render engine 222 may take intoaccount certain performance characteristics of the media processingsystem which will execute the user-defined media processing project. Inaccordance with this example implementation, render engine 222 mayaccess and analyze the system registry of the operating system, forexample, to ascertain the performance characteristics of hardware and/orsoftware elements of the computing system implementing the mediaprocessing system, and adjust the filter graph construction to improvethe perceived performance of the media processing system by the user.Nonetheless, there will always be a chance that a particular instance ofa filter graph will not be able to process the media stream fast enoughto provide the desired output at the desired time, i.e., processing ofthe media stream bogs down leading to delays at the rendering filter. Insuch a case, matrix switch 308 will recognize that it is not receivingmedia content at the appropriate project time, and may skip certainsections of the project in an effort to “catch-up” and continue theremainder of the project in real time. According to one implementation,when matrix switch 308 detects such a lag in processing, it will analyzethe degree of the lag and issue a seek command to the source (throughthe source processing chain) to a future point in the project, whereprocessing continues without processing any further content prior to theseeked point.

[0131] Thus, for the editing project depicted in FIG. 10, the processingdescribed above first builds a data structure (i.e. data structure 1200in FIG. 12) that represents the project in hierarchical space, and thenuses this data structure to define or create another data structure thatcan be utilized to program the matrix switch.

[0132]FIG. 19 is a flow diagram that describes steps in a method inaccordance with the described embodiment. The method can be implementedin any suitable hardware, software, firmware, or combination thereof. Inthe illustrated and described embodiment, the method is implemented insoftware.

[0133] Step 1900 provides a matrix switch. An exemplary matrix switch isdescribed above. Step 1902 defines a first data structure thatrepresents the editing project. Any suitable data structure can be used,as will be apparent to those of skill in the art. In the illustrated anddescribed embodiment, the data structure comprises a hierarchical treestructure having nodes that can represent tracks (having one or moresources), composites, transitions and effects. Step 1904 processes thefirst data structure to provide a second data structure that isconfigured to program the matrix switch. Any suitable data structure canbe utilized to implement the second data structure. In the illustratedand described embodiment, a grid structure is utilized. Exemplaryprocessing techniques for processing the first data structure to providethe second data structure are described above. Step 1906 then uses thesecond data structure to program the matrix switch.

[0134] Example Project with a Transition and an Effect

[0135] Consider project 2000 depicted in FIG. 20. In this project thereare three tracks, each of which contains a source, i.e. source A, B andC. This project includes an effect applied on source B and a transitionbetween sources B and C. The times are indicated as shown.

[0136] As the user creates their project, a data structure representingthe project is built. FIG. 21 shows an exemplary data structure in theform of a hierarchical tree 2100 that represents project 2000. There,the data structure includes three tracks, each of which contains one ofthe sources. The sources are arranged in the tree structure in the orderof their priority, starting with the lowest priority source on the leftand proceeding to the right. There is an effect (i.e. “Fx”) that isattached to or otherwise associated with source B. Additionally, thereis a transition attached to or otherwise associated with source C.

[0137] In building the grid for project 2000, the following rule isemployed for effects. An effect, in this example, is aone-input/one-output object that is applied to one object—in this casesource B. When the effect is inserted into the grid, it looks for anyone object beneath it in priority that has a desire to be routed to theprimary output of the matrix switch at the same time. When it finds asuitable object, it redirects that object's output from the matrixswitch's primary output to an output associated with the effect.

[0138] As an example, consider FIG. 22 and the grid 2200. At this pointin the processing of tree 2100, the rendering engine has incorporatedentries in the grid corresponding to sources A, B and the effect. It hasdone so by traversing the tree 2100 in the above-described way. In thisexample, the effect has already looked for an object beneath it inpriority that is competing for the primary output of the matrix switch.It found an entry for source B and then redirected B's grid entry to amatrix switch output pin that corresponds to the effect—here output pin1.

[0139] As the render engine 222 completes its traversal of tree 2100, itcompletes the grid. FIG. 23 shows a completed grid 2200. Processing ofthe grid after that which is indicated in FIG. 22 takes placesubstantially as described above with respect to the first example.Summarizing, this processing though: after the effect is entered intothe grid and processed as described above, the traversal of tree 2100next encounters the node associated with source C. Thus, a row is addedin the grid for source C and an entry is made to indicate that source Cdesires the output from t=12-16. Next, the tree traversal encounters thenode associated with the transition. Accordingly, a row is added to thegrid for the transition and a grid entry is made to indicate that thetransition desires the output from t=12-14. Now, as described above, thegrid is examined to find two entries, lower in priority than thetransition and located at the same tree level as the transition, thatcompete for the primary output of the matrix switch. Here, those entriescorrespond to the grid entries for the effect and source C that occurfrom t=12-14. These grid entries are thus redirected to output pins ofthe matrix switch 308 that correspond to the transition—here pins 2 and3 as indicated. Next, the grid is pruned which, in this example, removesa portion of the grid entry corresponding to source A for t=4-8 becauseof a conflict with the higher-priority entry for source B.

[0140]FIG. 24 shows the resultant matrix switch that has been built andconfigured as the grid was being processed above. At this point, thegrid can be used to program the matrix switch. From the grid picture, itis very easy to see how the matrix switch 308 is going to be programmed.Source A will be routed to the matrix switch's primary output (pin 0)from t=0-4; source B will be redirected to output pin 1 (effect) fromt=4-14 and the effect on B will be routed to the output pin 0 fromt=4-12. From t=12-14, the effect and source C will be routed to outputpins corresponding to the transition (pins 2 and 3) and, accordingly,during this time the transition (input pin 4) will be routed to theprimary output (output pin 0) of the matrix switch. From t=14-16, sourceC will be routed to the primary output of the matrix switch.

[0141] It will be appreciated that as the software, in this case therender engine 222, traverses the tree structure that represents aproject, it also builds the appropriate graphs and adds the appropriatefilters and graphs to the matrix switch. Thus, for example, as therender engine 222 encounters a tree node associated with source A, inaddition to adding an entry to the appropriate grid, the software buildsthe appropriate graphs (i.e. collection of linked filters), andassociates those filters with an input of the matrix switch. Similarly,when the render engine 222 encounters an effect node in the tree, thesoftware obtains an effect object or filter and associates it with theappropriate output of the matrix switch. Thus, in the above examples,traversal of the tree structure representing the project also enablesthe software to construct the appropriate graphs and obtain theappropriate objects and associate those items with the appropriateinputs/outputs of the matrix switch 308. Upon completion of the treetraversal and processing of the grid, an appropriate matrix switch hasbeen constructed, and the programming (i.e. timing) of inputs to outputsfor the matrix switch has been completed.

[0142] Treatment of “blanks” in a Project

[0143] There may be instances in a project when a user leaves a blank inthe project time line. During this blank period, no video or audio isscheduled for play.

[0144]FIG. 25 shows a project that has such a blank incorporatedtherein. If there is such a blank left in a project, the software isconfigured to obtain a “black” source and associate the source with thematrix switch at the appropriate input pin. The grid is then configuredwhen it is built to route the black source to the output at theappropriate times and fade from the black (and silent) source to thenext source at the appropriate times. The black source can also be usedif there is a transition placed on a source for which there is noadditional source from which to transition.

[0145] Audio Mixing

[0146] In the examples discussed above, sources comprising video streamswere discussed. In those examples, at any one time, only two videostreams were combined into one video stream. However, each project can,and usually does contain an audio component. Alternately, a project cancontain only an audio component. The audio component can typicallycomprise a number of different audio streams that are combined. Thediscussion below sets forth but one way of processing and combiningaudio streams.

[0147] In the illustrated example, there is no limit on the number ofaudio streams that can be combined at any one time.

[0148] Suppose, for example, there is an audio project that comprises 5tracks, A-E. FIG. 26 shows an exemplary project. The shaded portions ofeach track represent the time during which the track is not playing. So,for example, at t=0-4, tracks B, D, and E are mixed together and willplay. From t=4-10, tracks A-E are mixed together and will play, and thelike.

[0149]FIG. 27 shows the grid for this project at 2700. Since we aredealing with this composition now, all of the effects and transitionsincluding the audio mixing are only allowed to affect things in thiscomposition. Thus, there is the concept of a boundary 2702 that preventsany actions or operations in this composition from affecting any othergrid entries. Note that there are other entries in the grid and that thepresently-illustrated entries represent only those portions of theproject that relate to the audio mixing function.

[0150] Grid 2700 is essentially set up in a manner similar to thatdescribed above with respect to the video projects. That is, for eachtrack, a row is added to the grid and a grid entry is made for the timeperiod during which the source on that track desires to be routed to theprimary output of the matrix switch. In the present example, gridentries are made for sources A-E. Next, in the same way that atransition or effect was allocated a row in the grid, a “mix” element isallocated a row in the grid as shown and a grid entry is made toindicate that the mix element desires to be routed to the primary outputof the matrix switch for a period of time during which two or moresources compete for the matrix switch's primary output. Note that inthis embodiment, allocation of a grid row for the mix element can beimplied. Specifically, whereas in the case of a video project,overlapping sources simply result in playing the higher priority source(unless the user defines a transition between them), in the audio realm,overlapping sources are treated as an implicit request to mix them.Thus, the mix element is allocated a grid row any time there are two ormore overlapping sources.

[0151] Once the mix element is allocated into the grid, the grid isprocessed to redirect any conflicting source entries to matrix switchoutput pins that correspond to the mix element. In the above case,redirection of the grid entries starts with pin 3 and proceeds throughto pin 7. The corresponding matrix switch is shown in FIG. 28. Noticethat all of the sources are now redirected through the mix element whichis a multi-input/one output element. The mix element's output is fedback around and becomes input pin 15 of the matrix switch. All of theprogramming of the matrix switch is now reflected in the grid 2700.Specifically, for the indicated time period in the grid, each of thesources is routed to the mix element which, in turn, mixes theappropriate audio streams and presents them to the primary output pin 0of the matrix switch.

[0152] Compositions

[0153] There are situations that can arise when building an editingproject where it would be desirable to apply an effect or a transitionon just a subset of a particular project or track. Yet, there is nopracticable way to incorporate the desired effect or transition. In thepast, attempts to provide added flexibility for editing projects havebeen made in the form of so called “bounce tracks”, as will beappreciated and understood by those of skill in the art. The use ofbounce tracks essentially involves processing various video layers (i.e.tracks), writing or moving the processed layers or tracks to anotherlocation, and retrieving the processed layers when later needed foradditional processing with other layers or tracks. This type ofprocessing can be slow and inefficient.

[0154] To provide added flexibility and efficiency for multi-mediaediting projects, the notion of a composite or composition isintroduced. A composite or composition can be considered as arepresentation of an editing project as a single track. Recall thatediting projects can have one or more tracks, and each track can beassociated with one or more sources that can have effects applied onthem or transitions between them. In addition, compositions can benested inside one another.

[0155] Example Project with Composite

[0156] Consider, for example, FIG. 29 which illustrates an exemplaryproject 2900 having a composition 2902. In this example, composition2902 comprises sources B and C and a transition between B and C thatoccurs between t=12-14. This composition is treated as an individualtrack or layer. Project 2900 also includes a source A, and a transitionbetween source A and composition 2902 at t=4-8. It will be appreciatedthat compositions can be much more complicated than the illustratedcomposition, which is provided for exemplary purposes only. Compositionsare useful because they allow the grouping of a particular set ofoperations on one or more tracks. The operation set is performed on thegrouping, and does not affect tracks that are not within the grouping.To draw an analogy, a composition is similar in principle to amathematical parenthesis. Those operations that appear within theparenthesis are carried out in conjunction with those operations thatare intended to operate of the subject matter of the parenthesis. Theoperations within the parenthesis do not affect tracks that do notappear within the parenthesis.

[0157] In accordance with the processing that is described above inconnection with FIG. 19, a first data structure is defined thatrepresents the editing project. FIG. 30 shows an exemplary datastructure 3000 in the form of a hierarchical tree structure. In thisexample, group node 3002 includes two children—track node 3004 andcomposite node 3006. Track node 3004 is associated with source A.Composite node 3006 includes two children—track nodes 3008 and 3010 thatare respectively associated with sources B (3008 a) and C (3010 a). Atransition T2 (3012) is applied on source C and a transition T1 (3014)is applied on composition 3006.

[0158] Next, data structure 3000 is processed to provide a second datastructure that is configured to program the matrix switch. Note that asthe data structure is being programmed, a matrix switch is being builtand configured at the same time. In this example, the second datastructure comprises a grid structure that is assembled in much the sameway as was described above. There are, however, some differences and,for purposes of understanding, the complete evolution of the gridstructure is described here. In the discussion that follows, thecompleted matrix switch is shown in FIG. 38.

[0159] When the rendering engine initiates the depth-first,left-to-right traversal of data structure 3000, the first node itencounters is track node 3004 which is associated with source A. Thus, afirst row of the grid is defined and a grid entry is made thatrepresents the time period for which source A desires to be routed tothe matrix switch's primary output pin.

[0160]FIG. 31 shows the state of a grid 3100 after this first processingstep. Next the traversal of data structure 3000 encounters the compositenode 3006. The composite node is associated with two tracks—track 3008and track 3010. Track 3008 is associated with source B. Accordingly, asecond row of the grid is defined and a grid entry is made thatrepresents the time period for which source B desires to be routed tothe matrix switch's primary output pin. Additionally, since B is amember of a composition, meta-information is contained in the grid thatindicates that this grid row defines one boundary of the composition.This meta-information is graphically depicted with a bracket thatappears to the left of the grid row.

[0161]FIG. 32 shows the state of grid 3100 after this processing step.Next, the traversal of data structure 3000 encounters node 3010 which isassociated with source C. Thus, a third row of the grid is added and agrid entry is made that represents the time period for which source Cdesires to be routed to the matrix switch's primary output pin.

[0162]FIG. 33 shows the state of grid 3100 after this processing step.Notice that the bracket designating the composition now encompasses thegrid row associated with source C. The traversal next encounters node3012 which is the node associated with the second transition T2. Thus,as in the above example, a grid row is added for the transition and agrid entry is made that represents the time period for which thetransition desires to be routed to the matrix switch's primary outputpin.

[0163]FIG. 34 shows the state of grid 3100 after this processing step.Notice that the bracket designating the composition is now completed andencompasses grid row entries that correspond to sources B and C and thetransition between them. Recall from the examples above that atransition, in this example, is programmed to operate on two inputs andprovide a single output. In this instance, and because the transitionoccurs within a composition, the transition is constrained by a rulethat does not allow it to operate on any elements outside of thecomposition. Thus, starting at the transition entry and working backwardthrough the grid, entries at the same tree level and within thecomposition (as designated by the bracket) are examined to ascertainwhether they contain entries that indicate that they want to be routedto the output during the same time that the transition is to be routedto the output. Here, both of the entries for sources B and C haveportions that conflict with the transition's entry. Accordingly, thoseportions of the grid entries for sources B and C are redirected orchanged to correspond to output pins that are associated with atransition element that corresponds to transition T2.

[0164]FIG. 35 shows the state of grid 3100 after this processing step.The traversal next encounters node 3014 which is the node that isassociated with the transition that occurs between source A andcomposition 2902 (FIG. 29). Processing of this transition is similar toprocessing of the transition immediately above except for the fact thatthe transition does not occur within the composition. Because thetransition occurs between the composition and another source, one of theinputs for the transition will be the composition, and one of the inputswill be source A (which is outside of the composition). Thus, a grid rowis added for this transition and a grid entry is made that representsthe time period for which the transition desires to be routed to thematrix switch's primary output pin.

[0165]FIG. 36 shows the state of grid 3100 after this processing step.At this point then, the grid is examined for entries that conflict withthe entry for transition T1. One conflicting grid entry is found for therow that corresponds to source B (inside the composition) and one thatcorresponds to source A (outside the composition). Accordingly, thoseportions of the grid row that conflict with transition T1 are changed orredirected to have values that are associated with output pins of thematrix switch that are themselves associated with a transition elementT1. In this example, redirection causes an entry of “3” and “4” to beinserted as shown.

[0166]FIG. 37 shows the state of grid 3100 after this processing step.If necessary, a pruning operation would further ensure that the grid hasno competing entries for the primary output of the matrix switch. Theassociated input pin numbers of the matrix switch are shown to the leftof grid 3100.

[0167]FIG. 38 shows a suitably configured matrix switch that has beenbuild in accordance with the processing described above. Recall that, asdata structure 3000 (FIG. 30) is processed by the rendering engine, amatrix switch is built and configured in parallel with the building andprocessing of the grid structure that is utilized to program the matrixswitch. From the matrix switch and grid 3100 of FIG. 37, the programmingof the switch can be easily ascertained.

[0168]FIG. 38a shows an exemplary data structure that represents aproject that illustrates the usefulness of composites. In this example,the project can mathematically be represented as follows:

[0169] (Fx-noisy (A Tx-Blend B)) Tx-Blend C

[0170] Here, an effect (noisy) is applied to A blended with B, theresult of which is applied to a blend with C. The composite in thisexample allows the grouping of the things beneath it so that the effect(noisy), when it is applied, is applied to everything that is beneathit. Notice that without the composite node, there is no node where aneffect can be applied that will affect (A Tx-Blend B). Hence, in thisexample, operations that appear within the parenthesis are carried outon tracks that appear within the parenthesis. Those operations do notaffect tracks that are not within the parenthesis.

[0171]FIG. 39 is a flow diagram that described steps in a method inaccordance with one embodiment. The method can be implemented in anysuitable hardware, software, firmware, or combination thereof. In thepresently-described example, the method is implemented in software.

[0172] Step 3900 defines a multimedia editing project that includes atleast one composite. The composite represents multiple tracks as asingle track for purposes of the processing described just below. It isimportant to note that, in the processing described just below, andbecause of the use of composites, the extra processing that is requiredby bounce tracks is avoided (i.e. operating on two tracks, moving theoperation result to another location, and retrieving the operationresult when later needed). This reduces the processing time that isrequired to render a multi-media project. Step 3902 defines a first datastructure that represents the editing project. Any suitable datastructure can be utilized. In the present example, a data structure inthe form of a hierarchical tree is utilized. An exemplary tree is shownin FIG. 30. Step 3904 processes the first data structure to provide asecond data structure that is configured to program a matrix switch. Inthe illustrated example, the second data structure comprises a gridstructure. Exemplary processing is described in the context of FIGS.30-37. Step 3906 then programs the matrix switch using the second datastructure.

[0173] Implementing Dynamic Properties on Objects that Only SupportStatic Properties

[0174] Computer-implemented objects typically have properties associatedwith them. Many different types of properties can be associated withsuch objects. For example, consider a computer-implemented object in theform of a ball. The ball can have various properties such as color,direction of movement, speed, etc. These properties typically havevalues associated with them. For example, for the property “color”,property values might include blue, orange, red, and yellow. For theproperty “direction”, property values might include x-direction,y-direction, and z-direction.

[0175] Many objects have so-called static properties whose values canonly be set at a current time. In essence, these types of objects haveto be notified to change their property values currently. These types ofobjects are not pre-programmable to change their property values at somepoint in the future. Dynamic properties are properties whose values canchange at certain times. An object such as a ball that supports dynamicproperties can, for example, be pre-programmed to change its color,direction, speed, and the like, dynamically at any time in the future.That is, such objects can be preprogrammed to change their propertyvalues.

[0176] It would be desirable to be able to impart to objects thatsupport only static properties characteristics such that they behavelike they support dynamic properties. That is, it would be desirable tobe able to pre-program objects that support only static properties.

[0177]FIG. 40 is a block diagram of a system that can be utilized, inaccordance with the described embodiment, to impart to objects thatsupport only static properties, characteristics that enable them tobehave in a way that appears as if they support dynamic properties.

[0178] In the illustrated example, an object 4000 is provided andconstitutes any object that can support only static properties.Accordingly, a set of static properties 4002 is shown. It will beappreciated that any suitable number of objects such as object 4000 canbe provided. In addition, any number and/or type of suitable staticproperties can be supported by the object. For purposes of this example,however, assume that object 4000 represents a ball and that the ball hasproperties that include color, directions of movement and speed.

[0179] A helper object 4004 is provided and is used in connection withobject 4000 to, in a sense, imitate or simulate dynamic properties. Theresult of the interaction of the helper object 4004 and object 4000(which does not support dynamic properties) is an object 4000 thatappears as if it supports dynamic properties. In functioning to“imitate” dynamic properties, helper object 4004 can keep track of thetime, various property value changes that are to be made on object 4000,and the time at which the property value changes are to be made. Helperobject 4004 then notifies object 4000 at the appointed time, so that theproperty value can be changed.

[0180] Thus, in the described embodiment, an object that does notsupport dynamic properties is made to appear as if it supports dynamicproperties through the use of and interaction with a helper object. Thehelper object can maintain various property data that it then uses toeffect a property value change on its associated object. The propertydata can include, without limitation, the time when the change is totake effect, type of property value change, and how to effect thechange. In this manner, property values and changes in those values canbe pre-programmed.

[0181] In the illustrated and described embodiment, helper object 4004makes use of data structures that assist it in its job. For eachproperty whose values are to be changed, a set of structures is defined.Each structure comprises entries for time (i.e. the time that a propertyvalue change is to take place), a variant for the value of the property,and an entry for how the change is to take place (e.g. immediately, orin some interpolated fashion). Thus, in the FIG. 40 example, structurearray 4006 includes four sets of structures designated set 1, set 2, set3, and set 4. Each set is associated with a different property whosevalue can be changed on an object that supports only static properties.For example, one set each of structures might be associated with anobject's speed, color, and x/y direction of movement.

[0182] Consider the following example where object 4000 comprises aball. Consider that, for the ball, we want the helper object to causethe ball to be red at time 0, change immediately to blue at time 2, andblend into yellow at time 4. An exemplary data structure that can enablethis to be done is shown in FIG. 40. In this example, the data structure“(0, Red, Jump)” indicates that at time 0, the ball is to immediately(i.e. “jump to”) be red; the structure “(2, Blue, Jump)” indicates thatat time 2, the ball is to immediately be blue; and the structure “(4,Yellow, Interpolate)” indicates that at time 4, the ball is to change ina linear fashion (i.e. interpolate) into yellow from blue.

[0183] The helper object 4004 is programmed (or pre-programmed) with thevarious data structures that define the desired properties, their valuechanges, and the time and manner in which the value change is to takeplace. The helper object keeps track of the time, by for example, beingcalled or otherwise notified of the time. At the appropriate times, thehelper object, using the array of structures, calls the object whoseproperty values are desired to be changed so that the property valuescan be changed. In the present example, at time 0, the helper objectcalls object 4000 or a property object on the object itself, and causesthe property value for the color property to be set to “red”. At time 2,the helper object 4004 calls the object 4000 and causes the propertyvalue for the color property to be set to “blue”. At time 4, the helperobject calls the object 4000 and causes the property value for the colorproperty to interpolate from its previous value (i.e. blue) to thepresent value, i.e. yellow.

[0184] Thus, object 4000 appears as if it supports dynamic propertieswhen, in fact, it only supports static properties.

[0185] As another example, consider FIG. 41. In this example a ball 4100has directional properties that allows it to move within a coordinatesystem in both the x and y directions. The ball constitutes an objectthat supports only static properties, i.e. it cannot be pre-programmedfor movement in the future. The coordinates range from 0-200 in both thex and y directions as shown. Each property (movement in the y-directionand movement in the x-direction) is associated with a set of datastructures as indicated generally at 4102. In the present example, attime 0, the ball is located at x=0, y=100; at time 2, the ball jumps tox=100 and maintains its y position at 100. At time 4, the ball'sposition is interpolated (linearly) from x=100 to x=200, and from y=100to y=200.

[0186]FIG. 42 is a flow diagram that describes steps in a method inaccordance with the described embodiment. The method can be implementedin any suitable hardware, software, firmware or combination thereof. Inthe illustrated example, certain steps are implemented in software. Theillustrated method can relieve a top level application from the burdenof having to set object properties over and over again by keeping athread running. The top level application relies on the helper object toknow all of the properties that need to be changed or set and the timeat which to change or set the properties. The helper object will then dothe work involved with changing or setting the properties. This includeshaving a thread running and continuously setting or changing theproperties as instructed by the application. In performing its job, thehelper object can access a clock that the application is using so thatit knows what time it is so that it can, accordingly, set or change theright property.

[0187] Step 4200 provides one or more objects that support only staticproperties. These objects can be provided by an application executing ona suitable computer processor. Step 4202 provides at least oneprogrammable helper object. The helper object can be provided by theapplication program. The helper object is desirably configured to beused to effect property value changes on the objects that support onlystatic properties. Step 4204 programs the helper object with propertydata that is to be used by the helper object to effect one or moreproperty value changes on one or more of the objects that support onlystatic properties. This step can be implemented by an applicationprogram accepting input from a user that defines the property data.Specific examples of property data are given above and can include,without limitation, time when the property value change is to takeeffect, type of property value change, and how to effect the propertyvalue change. The property data can be organized in any suitable way.For example, in the discussion above, an array of data structures isused to hold the property data. Each structure array has one or moresets of data structures, and each set of data structures is associatedwith a property that is to be changed. It is to be appreciated, however,that any suitable method, structure, or technique of effecting theproperty changes can be used. Step 4206 then calls, with the helperobject(s), one or more of the objects that support only staticproperties to effect the property value change on the object(s). In theillustrated and described embodiment, the helper object is called withthe current time so that it knows when to call the objects whoseproperty values it desires to change.

[0188] Specific Example

[0189] The inventive helper object can be used in many contexts where itis desirable to impart to objects that support only static properties,characteristics or behavior that appears dynamic in nature. The helperobject can relieve the human programmer of a great burden which, in thepast, has required programmers to specifically provide the software code(including timing) to implement all of the property value changes. Oneparticularly useful context for application of the above-describedtechniques is in the area of multi-media and, in particular, the editingof multi-media projects.

[0190]FIG. 43 shows a matrix switch 4300 that has been built asdescribed above and, as shown, is configured to effect a transitionbetween two multi-media streams. To perform that task, a transitionobject 4302 is provided. The transition object is shown to include ahelper object 4304 having a structure array, and an object 4306 thatsupports only static properties. Object 4306 is the actual transformthat performs the desired transition and, in this example, isimplemented as a DirectX transform, although any suitable transform canbe used.

[0191] In this example, transition object 4302 has a property called“percent complete” and represents how complete the transition is for agiven project time. There are different percentages (i.e. values) ofcompletion that vary as the project time progresses. For example, attime 1, the transition may be 1% complete, at time 2 the transition maybe 3% complete and the like. Object 4306 supports only static propertiesand, hence, is called by the helper object 4304 during the course of atransition and told, by the helper object, to render a transition at aparticular percentage of completion. The helper object 4304 isprogrammed by data that is received from one or more of the matrixswitch, the grid and filter graphs that are assembled to complete theproject. This information is in a form that is not understood by theobject 4306 that performs the actual transition. Accordingly, the helperobject 4304 acts as an intermediary in that it receives programminginformation that describes when, how, and what properties and values areto be changed. The helper object then translates this programminginformation into instructions that are specific to and understood byobject 4306 that performs the actual transition.

[0192] Accordingly, in this example, helper object 4304 defines anintermediate level that provides the ability to simulate pre-programmeddynamic properties on an object (i.e. object 4306) that otherwise doesnot have the capability to have dynamic properties.

[0193] Conclusion

[0194] The described embodiments can be used to provide improvementsover previous multi-media editing systems. Various efficiencies areachieved that reduce the processing times and can thereby improve theuser experience when using multi-media project editing softwareapplications.

[0195] Although the invention has been described in language specific tostructural features and/or methodological steps, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or steps described. Rather, thespecific features and steps are disclosed as preferred forms ofimplementing the claimed invention.

1. A computer-implemented architecture comprising: one or more firstobjects that support only static properties; and one or more secondobjects associated with the one or more first objects and configured tocall the one or more first objects to effect one or more property valuechanges on the one or more first objects in a manner that makes the oneor more first objects appear as if they support dynamic properties. 2.The computer-implemented architecture of claim 1, wherein the one ormore second objects are configured to maintain property data that isused to call the one or more first objects.
 3. The computer-implementedarchitecture of claim 2, wherein the property data comprises at leastone property value change that is to be made.
 4. Thecomputer-implemented architecture of claim 2, wherein the property datacomprises a time at which a property value change is to be made.
 5. Thecomputer-implemented architecture of claim 2, wherein the property datacomprises how a property value change is to be made.
 6. Thecomputer-implemented architecture of claim 2, wherein the property datacomprises one or more of the following: at least one property valuechange that is to be made, a time at which a property value change is tobe made, and how a property value change is to be made.
 7. Thecomputer-implemented architecture of claim 2, wherein the property datacomprises at least one property value change that is to be made, a timeat which a property value change is to be made, and how a property valuechange is to be made.
 8. The computer-implemented architecture of claim1 further comprising one or more data structures associated with the oneor more second objects, individual data structures containing data thatis to be used by the one or more second objects to effect a propertyvalue change.
 9. The computer-implemented architecture of claim 8,wherein the one or more data structures comprise an array of one or moresets of data structures, each set of data structures being associatedwith a property that is to be changed and containing property data thatis to be used to change property values for a property.
 10. Thecomputer-implemented architecture of claim 9, wherein the property datacomprises at least one property value change that is to be made.
 11. Thecomputer-implemented architecture of claim 9, wherein the property datacomprises a time at which a property value change is to be made.
 12. Thecomputer-implemented architecture of claim 9, wherein the property datacomprises how a property value change is to be made.
 13. Thecomputer-implemented architecture of claim 9, wherein the property datacomprises at least one property value change that is to be made, a timeat which a property value change is to be made, and how a property valuechange is to be made.
 14. Software code embodied on a computer-readablemedium which, when executed by a computer, implements the system ofclaim
 1. 15. A multi-media editing application comprising thecomputer-implemented system of claim
 1. 16. A multi-media projectediting architecture comprising: one or more first objects that supportonly static properties, the one or more first objects being configuredto implement a transform associated with processing of a multi-mediaediting project; one or more second objects associated with the one ormore first objects and configured to call the one or more first objectsto effect one or more property value changes on the one or more firstobjects in a manner that makes the one or more first objects appear asif they support dynamic properties; and one or more data structuresassociated with the one or more second objects, individual datastructures containing property data that is to be used by the one ormore second objects to effect a property value change.
 17. Themulti-media project editing architecture of claim 16, wherein the one ormore data structures comprise an array of one or more sets of datastructures, each set of data structures being associated with a propertywhose values are to be changed and containing property data that is tobe used to change property values for a property.
 18. The multi-mediaproject editing architecture of claim 17, wherein the property datacomprises at least one value to which a property is to be changed. 19.The multi-media project editing architecture of claim 17, wherein theproperty data comprises a time at which at least one property value isto be changed.
 20. The multi-media project editing architecture of claim17, wherein the property data comprises how at least one property valueis to be changed.
 21. The multi-media project editing architecture ofclaim 17, wherein the property data comprises: at least one value towhich a property is to be changed, a time at which at least one propertyvalue is to be changed, and how at least one property value is to bechanged.
 22. Software code embodied on a computer-readable medium which,when executed by a computer, implements the system of claim
 16. 23. Amulti-media editing application comprising the computer-implementedsystem of claim
 16. 24. A multi-media project editing architecturecomprising: a software-implemented matrix switch having multiple inputpins and multiple output pins, the multiple input pins being routable tothe multiple output pins, the switch being configured to provide a datastream that represents a multi-media project; a data structureassociated with the matrix switch and configured for use in programmingthe matrix switch to provide a routing scheme for routing input pins tooutput pins; one or more first objects associated with the matrixswitch, the one or more first objects supporting only static propertiesassociated with rendering of a multi-media project; one or more secondobjects associated with the one or more first objects and configured tocall the one or more first objects to effect one or more property valuechanges on the one or more first objects in a manner that makes the oneor more first objects appear as if they support dynamic properties. 25.The multi-media project editing architecture of claim 24 furthercomprising one or more data structures associated with the one or moresecond objects, individual data structures containing data that is to beused by the one or more second objects to effect a property valuechange.
 26. The multi-media project editing architecture of claim 25,wherein the one or more data structures comprise an array of one or moresets of data structures, each set of data structures being associatedwith a property whose values is to be changed and containing propertydata that is to be used to change property values.
 27. The multi-mediaproject editing architecture of claim 26, wherein the property datacomprises a property value of a property that is to be changed.
 28. Themulti-media project editing architecture of claim 26, wherein theproperty data comprises a time at which a property value is to bechanged.
 29. The multi-media project editing architecture of claim 26,wherein the property data comprises how a property value is to bechanged.
 30. The multi-media project editing architecture of claim 26,wherein the property data comprises a property value of a property thatis to be changed, a time at which a property value is to be changed, andhow a property value is to be changed.
 31. A property value-changingmethod comprising: providing one or more objects that support onlystatic properties; providing one or more programmable objects configuredto effect property value changes on the objects that support only staticproperties; and effecting at least one property value change on the oneor more objects that support only static properties using the one ormore programmable objects.
 32. The method of claim 31 further comprisingprogramming the one or more programmable objects with property data thatis to be used by the one or more programmable objects to effect said atleast one property value change.
 33. The method of claim 32, wherein theproperty data comprises one or more property values that are to bechanged.
 34. The method of claim 32, wherein the property data comprisesa time at which a property value is to be changed.
 35. The method ofclaim 32, wherein the property data comprises how a property value is tobe changed.
 36. The method of claim 32, wherein the property datacomprises one or more property values that are to be changed, a time atwhich a property value is to be changed, and how a property value is tobe changed.
 37. The method of claim 32 further comprising organizing theproperty data in one or more data structures that are used by the one ormore programmable objects.
 38. The method of claim 32 further comprisingorganizing the property data in one or more data structures that areused by the one or more programmable objects, said organizing comprisesdefining an array of data structures, each array comprising one or moresets of structures and each set being associated with a property whosevalue can change.
 39. The method of claim 31, wherein said effectingcomprises calling the one or more objects that support only staticproperties with the one or more programmable objects.
 40. One or morecomputer-readable media having computer-readable instructions thereonwhich, when executed by a computer, implement the method of claim 31.41. A property value-changing method comprising: programming aprogrammable object with property data that defines when certainproperty value changes are to be made and what those property valuechanges are; calling, with the programmable object, one or more objectsthat do not support dynamic properties; and responsive to said calling,using the property data to effect a property value change on the one ormore objects that do not support dynamic properties.
 42. The method ofclaim 41 further comprising calling the programmable object with a timevalue, the programmable object using the time value to ascertain when tocall the one or more objects.
 43. The method of claim 41, wherein saidprogramming comprises arranging the property data in a data structurearray comprising one or more sets of data structures, each set of datastructures being associated with a property whose value is to bechanged.
 44. One or more computer-readable media havingcomputer-readable instructions thereon which, when executed by acomputer, implement the method of claim
 41. 45. One or morecomputer-readable media having computer-readable instructions thereonwhich, when executed by a computer, cause the computer to: provide oneor more objects that support only static properties; provide one or moreprogrammable objects configured to effect property value changes on theobjects that support only static properties; program the one or moreprogrammable objects with property data that is to be used by the one ormore programmable objects to effect said at least one property valuechange, the property data comprising: property value changes that are tobe made, time(s) at which property value changes are to be made, and howthe property value changes are to be made; and effect at least oneproperty value change on the one or more objects that support onlystatic properties by using the one or more programmable objects to callthe one or more objects that support only static properties.
 46. Aproperty value-changing method comprising: programming a programmableobject with property data that defines when certain property valuechanges are to be made and what those property value changes are, theproperty value changes being associated with rendering of a multi-mediaediting project; calling, with the programmable object, one or moreobjects that do not support dynamic properties; and responsive to saidcalling, using the property data to effect a property value change onthe one or more objects.
 47. The method of claim 46 further comprisingcalling the programmable object with a current time, the programmableobject using the current time to ascertain when to call the one or moreobjects.
 48. The method of claim 46, wherein said programming comprisesarranging the property data in a data structure array comprising one ormore sets of data structures, each set of data structures beingassociated with a property whose value is to be changed.
 49. The methodof claim 46, wherein the property data defines how the property valuechanges are to be made.
 50. One or more computer-readable media havingcomputer-readable instructions thereon which, when executed by acomputer, implement the method of claim
 46. 51. A propertyvalue-changing method comprising: providing one or more objects thatsupport only static properties; and simulating dynamic properties withthe one or more objects by changing one or more property values at apre-programmed time.
 52. The method of claim 51, wherein said simulatingcomprises pre-programming at least one property value change, a time atwhich the property value is to be changed, and a manner in which theproperty value change it to take place.
 53. The method of claim 52,wherein said pre-programming comprises pre-programming acomputer-implemented object to call the one or more objects at anappropriate time to change the one or more property values.
 54. Softwarecode comprising a multi-media project editing application configured toimplement the method of claim
 51. 55. A multi-media system comprising:an application program configured to enable a user to define amulti-media project in which multiple digital source streams can becombined; a software-implemented matrix switch having multiple inputpins and multiple output pins, the input pins being individuallyassociated with inputs that can compete, during a common time period,for a particular output pin that is associated with the matrix switch,the switch being configured to receive, at its input pins, digitalsource streams; a first data structure associated with the matrix switchand configured for use in programming the matrix switch to provide arouting scheme for routing input pins to output pins such that at anygiven time, only one input pin is routed to the particular output pin; asecond data structure associated with and different from the first datastructure, the second data structure representing a user-definedmulti-media project and being configured so that the first datastructure can be derived therefrom; one or more first objects associatedwith the matrix switch, the one or more first objects supporting onlystatic properties associated with rendering of a multi-media project;and one or more second objects associated with the one or more firstobjects and configured to call the one or more first objects to effectone or more property value changes on the one or more first objects in amanner that makes the one or more first objects appear as if theysupport dynamic properties.
 56. The multi-media system of claim 55further comprising one or more data structures associated with theprogrammable object(s), individual data structures containing data thatis to be used by the programmable object(s) to effect a property valuechange.
 57. The multi-media system of claim 56, wherein the one or moredata structures comprise an array of one or more sets of datastructures, each set of data structures being associated with a propertyvalue that is to be changed and containing property data that is to beused to change that property value.
 58. The multi-media system of claim56, wherein the one or more data structures comprise an array of one ormore sets of data structures, each set of data structures beingassociated with a property whose value is to be changed and containingproperty data that is to be used to change that property value, theproperty data comprising: a property value that is to be changed, a timeat which the property value is to be changed, and a manner in which theproperty value is to be changed.